GOSS: towards generalized open-set semantic segmentation

Abstract In this paper, we extend Open-set Semantic Segmentation (OSS) into a new image segmentation task called Generalized (GOSS). Previously, with well-known OSS, the intelligent agents only detect unknown regions without further processing, limiting their perception capacity of environment. It stands to reason that analysis detected pixels would be beneficial for agents’ decision-making. Therefore, propose GOSS, which holistically unifies abilities two well-defined tasks, i.e. OSS and generic segmentation. Specifically, GOSS classifies as belonging known classes, clusters (or groups) class are labelled such. We metric balances pixel classification clustering aspects evaluate newly expanded task. Moreover, build benchmark tests on existing datasets neural architectures baselines. Our experiments multiple benchmarks demonstrate effectiveness our Code is made available at https://github.com/JHome1/GOSS_Segmentor .